1. Technical Field
The present invention relates to a method and to a virtual penetrating mirror device for visualizing virtual objects from arbitrary view points within an augmented reality environment, with substantial use in medical and industrial applications.
2. Discussion of Related Art
The prior art related to the present invention is disclosed for example in CA2425075, US2002082498, U.S. Pat. No. 6,963,454, U.S. Pat. No. 6,999,239, US2005195279/US2002191003 and allows trailing of augmented reality systems with different capabilities regarding displays, tracking system and camera devices as to insert virtual objects in ureal scene.
However, the limitation of these methods in the mentioned patents is the lack of ability to provide intuitive control of the virtual objects.
The benefit of additional information due to augmented reality systems for interaction and navigation is disclosed in US2006284791 where multiple users can move within a virtual world and use a personal digital assistant (PDA) and a microphone for communicating information, which is not recognized by a single user.
However, entire examination of virtual entity can not be performed by a single user satisfactorily.
In US2007035563 an invention is disclosed using augmented reality to guide the user to a target object. Therefore the user receives initialization information, including a target location and a source location corresponding and gets navigational hints visualized in a screen space of the spatially enabled display.
However, additional abstract objects are required to guide the user to the target object, which in this case are available as non controllable features.
In WO03060830 an invention is described to make unseen hazards visible to increase preparedness dealing with the situation at hand. Information about hazards is given by computer-generated two- and three-dimensional graphical elements.
A limitation of WO03060830 is that the risky region visualized due to the virtual object can be seen only from one single direction and hence potential risk of the hazard cannot be fully estimated.
In US2003003430 method and apparatus are presented for an augmented reality-based fire fighter training system using realistic looking virtual fire, smoke, and extinguishing graphics to create a realistic scenario. The user is able to navigate a virtual water stream to extinguish virtual flames. Steam is visualized when water reaches fire to enhance realism of visual feedback due to interaction.
However, this patent does not disclose interactive methods to get an overview on virtual objects within a particular, hazardous situation that facilitates decision for further situational reaction.
In US2005251030 and US2005203380 system and method for surgical instrument placement using an image based navigation system is disclosed. Path to the object of interest in the patient can be interactively planed.
However, the augmented image is only generated for the viewpoint of the surgeon. Therefore regarding the surgical scenario disclosed in the mentioned patent, the possible surgeon's point of view on the operation site is restricted due to surrounding clinical staff, equipment in the operating room and the patient.
In DE10201116 a make-up mirror that can be attached to a PC monitor housing is described. Also an electronic mirror displayed on the monitor screen is mentioned, which is called “virtual mirror”.
However, this patent does not proclaim an augmented realty scenario where the user interacts with virtual objects integrated into a real scene.
US2003128182 discloses a virtual mirror. This patent proclaims a mirror providing additional virtual information concerning the environment of a car while driving. A virtual mirror is rendered on a computer graphics display or screen. The display substantially replicates the optical properties of a physical mirror. The display is provided on a host mobile body. Accessing a geospatial database, US2003128182 describes a virtual mirror displaying items that would be seen in a mirror with an unobstructed view based on information of the position of the mobile body and display data indicative of items in proximity to the mobile body.
The limitation of US2003128182 as being geospatially oriented is that it is not suitable for applications requiring interactivity in personal space as it is the case for surgical operations and service and/or maintenance procedures on industrial components or products.
The proposed invention is not applicable to a medical or an industrial augmented reality environment.
A further limitation of US2003128182 is that the virtual object can be seen only from one single direction related to the actual position of the mobile body. The proposed mirror in US2003128182 therefore does not allow the virtual object to be seen from any arbitrary point of view, for example from the beneath, upon or behind in regard to the viewing direction.
One further limitation of US2003128182 is that the reflected object is only an approximation of the item intended to be mirrored.
The increasing amount of data provided by medical 3D imaging devices to surgeons makes new visualization technology necessary that reveal more information at a glance other than a stack of 2D slices. Visualizing-3D imaging data is less cumbersome than browsing through these stacks. As opposed to 2D images 3D data does not have a direct representation on a screen, hence it needs a recalculation to visualize it.
Ideally, the visualizations are embedded into the context of their use and not far away from the operation site as in current practice.
In order to take full advantage of the spatial relations in the 3D data, visualizations must convey the spatial information. The information perpendicular to the viewing direction is commonly preserved, however the absolute and relative depth information is gone on an ordinary 2D display. A promising technology for overcoming some of these shortcoming is augmented reality.
Augmented Reality for intra-operative 3D-visualization and navigation has been a subject of intensive research and development during the last decade. For example in U.S. Pat. No. 6,891,518 and U.S. Pat. No. 6,919,867 a basic system and method for visualization in an augmented reality environment is disclosed.